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1 Introduction 

The study of mutual entropy (information) and capacity in classical system 
was extensively done after Shannon by several authors like Kolmogorov |T^ 



and Gelfand [0. In quantum systems, there have been several definitions of 
the mutual entropy for classical input and quantum output f^, |], ||, |T^. In 



1983, the author defined the fully quantum mechanical mutual entropy 



by means of the relative entropy of Umegaki |3q], and he extended it |23] to 



general quantum systems by the relative entropy of Araki B and Uhlmann 



3^ . When the author introduced the quantum mutual entropy, he did not 
indicate that it contains other definitions of the mutual entropy including 
classical one, so that there exist several misunderstandings for the use of 
the mutual entropy (information) to compute the capacity of quantum chan- 
nels. Therefore in this note we point out that our quantum mutual entropy 
generalizes others and where the misuse occurs. 



2 Mutual Entropy 

The quantum mutual entropy was introduced in |^T| for a quantum input 
and quantum output, namely, for a purely quantum channel, and it was gen- 
eralized for a general quantum system described by C*-algebraic terminology 
p3| . We here review the mutual entropy in usual quantum system described 
by a Hilbert space. 

Let H he a Hilbert space for an input space, B(7i) be the set of all 
bounded linear operators on Ti and iS(7i) be the set of all density operators 

on Ti. An output space is described by another Hilbert space , but often 
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= 7"^. A channel from the input system to the output system is a mapping 
A* from S{H) to S(j-l) A channel A* is said to be completely positive 
if the dual map A satisfies the following condition: ^k,j=i '^kMBk^j)^j ^ 
for any n gN and any Aj G BiTi), Bj G B{j-i). This condition is not strong 



at all because almost all physical transformations satisfy it [0, . 

An input state p G SiTi) is sent to the output system through a channel 

A*, so that the output state is written as P= A* p. Then it is important to 
ask how much information of p is correctly sent to the output state A* p. This 
amount of information transmitted from input to output is expressed by the 
mutual entropy. 

In order to define the mutual entropy, we first mention the entropy of a 



quantum state introduced by von Neumann |T^. For a state p, there exists 
a unique spectral decomposition 

p = SfcA,Pfc, (2.1) 

where is an eigenvalue of p and Pk is the associated projection for each 
Afc. The projection is not one-dimensional when A^ is degenerated, so that 
the spectral decomposition can be further decomposed into one- dimensional 



projections. Such a decomposition is called a Schatten decomposition |l33 
namely, 

p = EfcAfc^fc, (2.2) 

where E^. is the one-dimensional projection associated with and the de- 
generated eigenvalue A^ repeats dimP^ times; for instance, if the eigenvalue 
Aihas the degeneracy 3, then Ai = A2 = A3 < A4. This Schatten decom- 
position is not unique unless every eigenvalue is non-degenerated. Then the 
entropy (von Neumann entropy) S (p) of a state p is defined by 

S{p) = -trp\ogp, (2.3) 
which equals to the Shannon entropy of the probabihty distribution {\k} '■ 

S{p) = -Y.^^^''^^^- (2.4) 

k 

The quantum mutual entropy was introduced on the basis of the above 
von Neumann entropy for purely quantum communication processes. The 
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mutual entropy depends on an input state p and a channel A*, so it is denoted 
by / (p; A*), which should satisfy the following conditions: 

(1) The quantum mutual entropy is well-matched to the von Neumann 
entropy. Furthermore, if a channel is trivial, i.e.. A* = identity map, then 
the mutual entropy equals to the von Neumann entropy: / (p; id) = S (p). 

(2) When the system is classical, the quantum mutual entropy reduces to 
classical one. 

(3) Shannon's fundamental inequality ||3^ 0< /(p;A*) < S (p) is held. 



Before mentioning the quantum mutual entropy, we briefly review the 
classical mutual entropy 0. Let , (fi,JF)be an input and output 

measurable spaces, respectively, and P , P (fi) are the corresponding sets 
of all probability measures (states) on Q and Q , respectively. A channel A* is 
a mapping from P (i7) to P (i7) and its dual A is a map from the set B (f2) of 
all Baire measurable functions on fl to B . For an input state p E P (fl) , 
the output state JI = A*p and the joint state (probability measure) $ is given 
by 

$(Qx Q)= [_A{lQ)dp, QeT, QeT, (2.5) 

where 1q is the characteristic function on Q : 1q (tu) = | ^ 1^ ^ nl ■ "^^^ 
classical entropy, relative entropy and mutual entropy are defined as follows: 

5 (p) = sup I - p (Afc) log p{Ak);{Ak}eV (Q) | , (2.6) 

l^iAk 



S (p, ^) = sup |g p {A,) log {A,} e V {Q) 



(2.7) 



/(p;A*) = ^(<l>,p® A*p), (2.8) 

where V (Q) is the set of all finite partitions on Q, that is, {Ak} G V (Q) iff 
Ak e with Ak nAj = (D{kj^ j)and U^^i^fc = Q. 

The quantum mutual entropy is defined as follows: In order to define the 
quantum mutual entropy, we need the quantum relative entropy and the joint 
state (it is called "compound state" in the sequel) describing the correlation 
between an input state p and the output state A*p through a channel A*. 
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A finite partition of Q in classical case corresponds to an orthogonal decom- 
position {-Efc} of the identity operator I of 7i in quantum case because the 
set of all orthogonal projections is considered to make an event system in a 
quantum system. It is known ||23| that the following equality holds 



sup | - ^trpEfc logtrpEk, {Ek}j= -trp log p, 

and the supremum is attained when {E^} is a Schatten decomposition of p. 
Therefore the Schatten decomposition is used to define the compound state 
and the quantum mutual entropy. 

The compound state aE (corresponding to joint state in CS) of p and A*p 
was introduced in [pT] , p^ , which is given by 



J2>^kEk0A*Ek, (2.9) 



where E stands for a Schatten decomposition {Ek} of p, so that the com- 
pound state depends on how we decompose the state p into basic states 
(elementary events), in other words, how to see the input state. 



The relative entropy for two states p and a is defined by Umegaki |35 
and Lindblad 11^, which is written as 



^ / cr) — / (logP ~ logcr) (when ranp C rana) ,^ . 

[ oo (otherwise) 

Then we can define the mutual entropy by means of the compound state 



and the relative entropy |21], that is. 



/ (p; A*) = sup {S {aE, p ® A^p) ; E = {E,}} , (2.11) 

where the supremum is taken over all Schatten decompositions because this 
decomposition is not unique generally. Some computations reduce it to the 
following form: 

/ (p; A*) = sup ^Y^^kS {A*Ek, A*p) ; E = {Ek} | . (2.12) 
This mutual entropy satisfies all conditions (1)~(3) mentioned above. 
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When the input system is classical, an input state p is given by a prob- 
ability distribution or a probability measure, in either case, the Schatten 
decomposition of p is unique, namely, for the case of probability distribution 
; P = {Afc} , 

P = J]Afc4, (2.13) 

k 

where 6k is the delta measure, that is, 

4(j)=4,, = {;s:3j,vj. (2.14) 

Therefore for any channel A*, the mutual entropy becomes 

/(p;A*) = 5^Afc5(A*4,AV), (2.15) 

k 

which equals to the following usual expression of Shannon when the minus 
is well-defined: 

/ (p; A*) = S (A*p) - J2 >^kS {A*5k) . (2.16) 

k 

The above equality has been taken as the definition of the mutual entropy 
for a classical-quantum channel |^ |], ||, ^ p!^ . 

Note that the definition (2.12) of the mutual entropy is written as 
/ (p; A*) = sup <^ ^ \kS (AVfc, A*p) ;p = J2 ^^P^ ^ ^° ('^) f ' 

y k k J 

where Fo (p) is the set of all orthogonal finite decompositions of p. Here 
Pk is orthogonal to pj (denoted by pk -L Pj) means that the range of pk is 
orthogonal to that of pj. The equality is easily proved as follows: Put 

// (p; A*) = sup j ^ \kS {A*pk, A*p) ; P = ^''Pk ^ (p) [ . 

I fc k J 

The inequality / (p; A*) < // (p; A*) is obvious. Let us prove the converse. 
Each Pk in an orthogonal decomposition of p is further decomposed into 
one dimensional projections; pk = ^jpf^Ej, a Schatten decomposition 
of Pk- From the following equalities of the relative entropy ^: (1) 
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S {ap,ha) = aS{p,a) — alog^, for any positive number a, 6; (2) pi ± p2 
=^ S {pi + p2, 0") = 5* (pi, cr) + 5* (p2, a) , we have 

(A*p,, A*p) = Efc,, Afcpf ^5 (A^Ef U*p) + E.,, A.pf ^ logpf ) 
<E.,A,pf5(A*Ef,A*p), 
which imphes the converse inequahty / (p; A*) > If (p; A*) because 
Sfc j ^kpf'' E^j'^ is a Schatten decomposition of p. Thus / (p; A*) = J/ (p; A*) . 

More general formulation of the mutual entropy for general quantum sys- 
tems was done |TU| in C*dynamical system by using Araki's or Uhlmann's 
relative entropy 0, ^ |2^. This general mutual entropy contains all other 
cases including measure theoretic definition of Gelfand and Yaglom . 

The mutual entropy is a measure for not only information transmission 
but also description of state change, so that this quantity can be applied to 
several topics Q, |, p3|, |1 p^, p. 



3 Communication Processes 



We discuss communication processes in this section]^, |2^. Let A = {ai,a2 



■, ttn} be a set of certain alphabets and be the infinite direct product of A : 
Q = A^ = U'^^A calling a message space. In order to send a information 
written by an element of this message space to a receiver, we often need to 
transfer the message into a proper form for a communication channel. This 
change of a message is called a coding. Precisely, a coding is a measurable 
one to one map ^ from f2 to a proper space X . For instance, we have the 
following codings: (1) When a message is expressed by binary symbol and 
1, such a coding is a map from Q to {0, 1}'' . (2) A message expressed by 0,1 
sequence in (1) is represented by an electric signal. (3) Instead of an electric 
signal, we use optical signal. Coding is a combination of several maps like 
the above (1) and (2), (3). One of main targets of the coding theory is to find 
the most efficient coding and also decoding for information transmission. 

Let {Q, J^fi, P{Q)) be an input probability space and X be the coded 
input space. This space X may be a classical object or a quantum object. 
For instance, X is a Hilbert space Ti of a quantum system, then the coded 
input system is described by {B(H),S(H))of Sec. 2. 

An output system is similarly described as the input system: The coded 

output space is denoted by X and the decoded output space is Q made by 
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another alphabets. An transmission (map) from X to X is described by a 
channel reflecting all properties of a physical device, which is denoted by 

7 here. With a decoding ^, the whole information transmission process is 
written as 

n^x ^x^n . (3.1) 

That is, a message G is coded to ^ (cj) and it is sent to the output system 
through a channel 7, then the output coded message becomes 70^ [cu) and 

it is decoded to ^ 07 o ^ (1^) at a receiver. 

This transmission process is mathematically set as follows: M messages 
are sent to a receiver and the kth message u)^^^ occurs with the probabil- 
ity Ajfc. Then the occurrence probability of each message in the sequence 
(u;^^\u;^'^\ ■ ■ •, a;(^^)of M messages is denoted by p = {Xk} , which is a state 
in a classical system. If ^ is a classical coding, then ^ (cu) is a classical object 
such as an electric pulse. If ^ is a quantum coding, then ^ (c<;) is a quantum 
object (state) such as a coherent state. Here we consider such a quantum 
coding, that is, ^ (o;*^*^)) is a quantum state, and we denote ^ (a;^'^)) by a^. 
Thus the coded state for the sequence • • -,0;*^^)) is written as 

(7 = ^Afe(7jfc. (3.2) 

k 

This state is transmitted through a channel 7. This channel is expressed by 
a completely positive mapping F*, in the sense of Sec.l, from the state space 

of X to that of X , hence the output coded quantum state a is r*a. Since 
the information transmission process can be understood as a process of state 

(probability) change, when ft and Q, are classical and X and X are quantum, 
the process (3.1) is written as 

p{n)^s (H) ^s{n)^ p{n), (3.3) 

where S* (resp.S ) is the channel corresponding to the coding ^ (resp.^ ) and 
S (Ti) (resp.»S( Ti) ) is the set of all density operators (states) on H {resp.Ti 
)• 

We have to be care to study the objects in the above transmission process 
(3.1) or (3.3). Namely, we have to make clear which object is going to study. 
For instance, if we want to know the information capacity of a quantum 
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channel 7(= F*), then we have to take X so as to describe a quantum system 
Ukc a Hilbcrt space and wc need to start the study from a quantum state in 
quantum space X not from a classical state associated to a message. If we like 
to know the capacity of the whole process including a coding and a decoding, 

which means the capacity of a channel ^ 07 o ^(=S o F* o S*), then we have 
to start from a classical state. In any case, when we concern the capacity of 
channel, we have only to take the supremum of the mutual entropy / (p; A*) 
over a quantum or classical state p in a proper set determined by what we 
like to study with a channel A*. We explain this more precisely in the next 
section. 

4 Channel Capacity 

We discuss two types of channel capacity in communication processes, namely, 
the capacity of a quantum channel F* and that of a classical (classical- 

quantum-classical) channel S o F* o S*. 

(1) Capacity of quantum channel: The capacity of a quantum channel is 
the ability of information transmission of the channel itself, so that it does 
not depend on how to code a message being treated as a classical object and 
we have to start from an arbitrary quantum state and find the supremum of 
the mutual entropy. One often makes a mistake in this point. For example, 
one starts from the coding of a message and compute the supremum of the 
mutual entropy and he says that the supremum is the capacity of a quantum 
channel, which is not correct. Even when his coding is a quantum coding 
and he sends the coded message to a receiver through a quantum channel, if 
he starts from a classical state, then his capacity is not the capacity of the 
quantum channel itself. In his case, usual Shannon's theory is applied because 
he can easily compute the conditional distribution by a usual (classical) way. 
His supremum is the capacity of a classical-quantum-classical channel, and 
it is in the second category discussed below. 

The capacity of a quantum channel F* is defined as follows: Let «So(c 
S{T-C)) be the set of all states prepared for expression of information. Then 
the capacity of the channel F* with respect to Sq is defined by 

C^"(r*) = sup{/(p;F*);pG5o}. (4.1) 
Here /(p;F*) is the mutual entropy given in (2.11) or (2.12) with A* = F*. 
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When So = S{n) , C-^^^) (r*) is denoted by C (F*) for simplicity. In ^ [T^, 
we also considered the pseudo-quantum capacity Cp (F*) defined by (4.1) 
with the pseudo-mutual entropy Ip (p; F*) where the supremum is taken over 
all finite decompositions instead of all orthogonal pure decompositions: 

Ip (p; F*) = sup < \kS (T*pk, T*p) ; p = XkPk, finite decomposition 

I A: k 

(4.2) 

However the pseudo-mutual entropy is not well-matched to the conditions 



explained in Sec. 2, and it is difficult to be computed numerically |30|. From 
the monotonicity of the mutual entropy E^, we have 



< C^^ in < (F*) < sup{^(p);p e So}. 

(2) Capacity of classical-quantum-classical channel: The capacity of C-Q- 

C channel S o F* o H* is the capacity of the information transmission process 
starting from the coding of messages, therefore it can be considered as the 
capacity including a coding (and a decoding). As is discussed in Sec. 3, an 
input state p is the probability distribution {A^} of messages, and its Schatten 
decomposition is unique as (2.13), so the mutual entropy is written by (2.15): 



/ (p; S* o F* o S*) = J2 >^kS (e o F* o E*5k, E oT*o H*p) . (4.3) 

k 

If the coding S* is a quantum coding, then E*6k is expressed by a quantum 
state. Let denote the coded quantum state by ak and put a = E*p = 
Ylk ^kO'k- Then the above mutual entropy is written as 

/ (p; S* o F* o S*) = J2 (e o FVfc, E o FV) . (4.4) 

k 

This is the expression of the mutual entropy of the whole information trans- 
mission process starting from a coding of classical messages. Hence the ca- 
pacity of C-Q-C channel is 

(e o F* o S*) = sup{/ (p; E oT*oE*);pe Pq}, (4.5) 

where Po{c P{^)) is the set of all probability distributions prepared for 
input (a-priori) states (distributions or probability measures). Moreover the 
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capacity for coding is found by taking the supremum of the mutual entropy 
(4.4) over all probability distributions and all codings S*: 

Cfo (S* o r*) = sup{/ (p; E oT*oE*y,pe Po, 5*}. (4.6) 
The last capacity is for both coding and decoding and it is given by 

C5 ( r*) = sup{/ (p; EoT*oE*y,pe Po, S*, E}. (4.7) 

These capacities , do not measure the ability of the quantum channel 
r* itself, but measure the ability of F* through the coding and decoding. 
Remark that XkS{r*ak) is finite, then (4.4) becomes 

/ (p; H* o r* o S*) = SiE orV) - ^ XkSiE orVfc). (4.8) 

k 

Further, if p is a probability measure having a density function /(A) and each 
A corresponds to a quantum coded state cr(A), then cr = J /(A) a{X)dX and 



/ (p; o r* o S*) = SiE orV) - J fiX)SiE orV(A))ciA, (4.9) 
which is less than 

S{T*a) - J f{X)S{T*a{X))dX. 
The above bound is called Holevo bound, and it is computed in several 



cases 



29, 36 



The above three capacities , " , satisfy the following inequalities 

< (e o r* o s*) < cf' (e o r*) < c^' ( r*) < sup {Sip); p e Po} 

where S{p) is not the von Neumann entropy but the Shannon entropy: - 
AfclogAfc. 

The capacities (4.1), (4. 6), (4. 7) and (4.8) are generally different. Some 
misunderstandings occur due to forgetting which channel is considered. That 
is, we have to make clear what kind of the ability, the capacity of a quantum 
channel itself or that of a classical-quantum(-classical ) channel or that of a 
coding free, is considered. 
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